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PREFACE 


This  report  was  created  for  the  F-16  Aircrew  Training  De¬ 
velopment  Project  contract  no.  F02604-79-C8875  for  the  Tactical 
Air  Command  to  comply  with  the  requirements  of  CDRL  no.  6013. 

Tne  project  entailed  the  design  and  development  of  an  instruc¬ 
tional  system  for  the  F-16  RTU  and  instructor  pilots.  During  the 
course  of  the  project,  a  series  of  development  reports  was  issued 
describing  processes  and  products.  A  list  of  those  reports 
follows  this  page.  The  user  is  referred  to  Report  No.  34,  A 
Users  Guide  to  the  F-16  Training  Development  Reports,  for  an 
overview  and  explanation  of  the  series,  and  Report  No.  35,  F-16 
Final  Report,  for  an  overview  of  the  Instructional  System  De¬ 
velopment  Project. 
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EXECUTIVE  SUMMARY 


Objective  hierarchy  analysis  is  the  process  of  identifying 
the  specific  instructional  performances  students  must  achieve  in 
order  to  master  the  behaviors  indicated  in  a  task  listing.  The 
objectives  identified  in  the  hierarchy  analysis  should  include 
all  of  the  categories  of  behavior  required  to  prepare  students 
for  on-the-job  performance.  These  categories  include  verbal 
behaviors,  motor  behaviors,  and  intellectual  skills.  The 
objective  hierarchy  analysis  procedure  used  in  the  F-16  project 
included  the  following  steps: 

1.  A  task  is  selected. 

2.  Subtasks  are  listed. 

3.  Decision-making  behaviors  (classification  and  rule  using) 
are  identified. 

4.  Motor  control  behaviors  are  identified. 

5.  Verbal  support  (summarization)  requirements  are  identified. 

6.  Minor  tasks  are  grouped  for  instruction. 

7.  Dangerous  and  excessively  expensive  behaviors  are  scaled 
back. 

8.  Knowledge-base  requirements  are  determined. 

9.  Remaining  tasks  are  eliminated  (pruning). 

10.  A  new  task  is  selected  and  processing  begins  again. 

In  order  to  facilitate  the  updating  anc  revision  of  the 
objectives  hierarchies,  the  maintenance  responsibilities  of 
various  ISD  personnel  and  the  related  support  requirements  are 
described . 
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OBJECTIVE  HIERARCHY  ANALYSIS 
METHODOLOGY  REPORT 


1 . 0  INTRODUCTION 


This  report  describes  the  objective  hierarchy  analysis  meth¬ 
odology  used  during  development  of  the  F-16  objectives  hierarch¬ 
ies  and  references  the  methodology  recommended  for  its  mainte¬ 
nance  throughout  the  life  of  the  F-16  instructional  system.  This 
report  is  related  to  other  project  reports  which  describe  devel¬ 
opment  methodology  and  especially  to  those  reports  related  to 
task  analysis  and  criterion-referenced  objective  writing  and 
update.  (See  project  reports  no.  7,  "Task  Analysis  Methodology 
Report",  and  no.  5,  "Derivation,  Formating  and  Use  of  Criterion- 
referenced  Objectives  (CROs)  and  Criterion-referenced  Tests 
(CRTs).")  These  reports  stress  the  importance  of  the  instruc¬ 
tional  system  manager's  understanding  the  methods  used  in 
analysis  and  the  importance  of  implementing  them  in  a  continual 
update  process  aimed  at  preserving  both  the  currency  of  the 
analysis  and  the  currency  of  the  instructional  materials  based  on 
the  analysis. 

This  report  is  written  in  four  sections  describing  the 
objectives  hierarchy  process  used  during  system  development  and 
the  maintenance  procedure  for  the  hierarchies.  Section  2.0  of 
the  report  describes  the  purpose  and  characteristics  of  the 
objectives  hierarchies  and  the  relation  of  the  analysis  process 
to  other  instructional  development  processes.  Section  3.0 
describes  in  a  step-by-step  fashion  the  objective  hierarchy 
analysis  procedure  used  for  generation  of  the  F-16  pilot  and 
instructor  pilot  hierarchies.  Section  4.0  summarizes  the  need 
for  continual  update  and  maintenance  of  the  hierarchies,  refers 
to  a  generalized  procedure  for  that  published  in  another  project 
document,  and  recommends  maintenance  organizational  roles  for 
accomplishing  the  task  as  well  as  stating  required  items  of 
logistic  support. 
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2.0  RATIONALE 


This  section  describes  the  purpose  of  objectives  hierarchy 
analysis,  the  desirable  qualities  of  analysis,  and  the  relation 
of  objectives  hierarchy  analysis  to  other  instructional  develop¬ 
ment  processes. 


2 . 1  Purpose 

Objectives  hierarchy  analysis  is  a  process  which  begins  with 
the  product  of  task  analysis  (the  task  listing  or  inventory)  to 
identify  in  a  logical,  consistent  way  the  individual  instruc¬ 
tional  goals  students  must  reach  as  they  progress  toward  mastery 
of  behaviors  contained  in  the  task  listing.  The  task  listing 
behaviors  used  as  the  beginning  point  for  analysis  usually  repre¬ 
sent  intermediate  level  course  behavior  goals,  so  the  objectives 
identified  during  analysis  represent  a  lowest-level  set  of 
instructional  goals  for  the  training  scheme.  The  objectives 
identified  through  analysis  should  include  all  of  the  varieties 
of  behavior  required  to  prepare  students  for  performance,  includ¬ 
ing  verbal  behaviors,  motor  behaviors,  and  intellectual  skills 
required  for  performance. 

Objectives  hierarchy  analysis  is  closely  related  to  task 
analysis  but  possesses  important  similarities  and  differences. 
First,  objectives  hierarchies  are  similar  to  task  analysis 
because  both  processes  are  performed  for  similar  purposes.  Both 
are  designed  to  identify  only  that  minimum  of  content  and  behav¬ 
ior  required  for  training  while  excluding  the  unnecessary.  As 
well,  both  are  designed  to  exhaustively  identify  all  those  things 
which  are  necessary.  Both  processes  consist  of:  (1)  A  set  of 
guidelines  and  procedures  for  guiding  the  thought  and  decision¬ 
making  processes  of  the  developer  and  (2)  a  notation  system  for 
capturing  the  thought  and  decision  produced  using  the  guidelines 
on  paper  for  later  reference  and  examination.  Neither  is  the 
substitute  for  thought,  but  rather  a  channel  to  guide  it  and  a 
means  of  representing  the  resulting  decisions. 

Objectives  hierarchy  analysis  is  also  related  to  task 
analysis  in  that  both  may  be  displayed  in  the  same  notational 
system,  a  hierarchy  diagram.  Since  the  objectives  hierarchy 
analysis  process  begins  with  the  task  analysis  process  product 
and  continues  on,  the  resultant  product  of  both  processes  might 
resemble  the  diagram  in  Figure  1.  The  diagram  shows  that  not 
only  do  objectives  appear  appended  to  tasks  at  the  lowest  levels 
of  the  task  analysis  but  appear  within  the  body  of  the  analysis 
as  well. 
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Representation  of  the  Relationship  Between  Task 
id  Objectives  Hierarchy  Analysis 


Though  the  representation  of  the  two  analyses  may  be  the 
same,  the  processes  and  logic  of  each  are  vastly  different.  Task 
analysis  is  a  logical  decomposition  process  of  tasks  into  smaller 
tasks.  This  is  achieved  by  identifying  either  the  steps  in 
sequence  of  the  larger  task  or  sub-variet ies  of  the  larger  task. 
Objectives  hierarchy  analysis,  in  contrast,  proceeds  by  a  logic 
which  takes  the  list  of  sub-tasks  for  a  given  task  and  asks 
questions  relative  to  the  specific  training  difficulty  and 
strategy  for  each.  Because  of  this  difference  in  logical 
processes  between  the  two  analyses,  many  find  task  analysis  to  be 
quite  easy  while  finding  objectives  hierarchy  analysis  to  be 
difficult.  The  reason  is  most  likely  a  familiarity  with  the 
logic  of  the  first,  which  is  simple,  and  a  lack  of  familiarity 
with  the  logic  of  the  second,  which  is  also  conceptually  simple 
but  not  as  common . 

The  present  method  is  intended  to  assist  the  technologist  by 
providing  an  explicit  procedure.  With  a  modicum  of  training  and 
supervision  using  this  method  the  nonscientist  can  participate 
effectively  in  analysis  and  produce  a  logically  consistent 
product.  A  product  serviceable  to  developer  and  student  alike. 

The  process  outlined  in  this  report  is  closely  related  to  a 
widely  used  method  of  analysis  proposed  by  Gagne  (1968)  and  has 
the  same  end  in  mind.  In  Gagne's  method,  tasks  from  the  task 
listing  are  analyzed  through  recursive  asking  of  the  question, 
"What  must  a  person  be  able  to  know  and  do  to  be  able  to  perform 
this  task?".  That  procedure  has  been  much  used  and  misused, 
perhaps  because  of  its  lack  of  procedural  specificity  and  its 
reliance  on  the  bias  and  insight  of  the  analyst.  What  the 
analyst  feels  the  student  "must  know"  encourages  the  injection  of 
subjectivity  into  the  process.  A  high  dependence  upon  creative 
insight  may  be  highly  desirable  for  the  scientist  seeking  new 
techniques,  new  theoretical  systems,  and  new  styles  of  analysis 
or  to  the  expert  and  experienced  analyst,  but  it  is  not  satis¬ 
factory  for  the  technologist  who,  with  less  training,  is  required 
to  apply  the  process  with  some  uniformity  in  the  outcome.  As  a 
consequence  of  the  lack  of  procedural  guidance  for  this  group, 
analysis  products  tend  to  be  highly  variable,  and  dependable 
analyses  must  be  carried  out  in  the  main  by  highly  trained 
persons . 


2.2  Relation  of  Objectives  Hierarchy  Analysis  to  other 

Instructional  Development  Processes 

Because  objectives  hierarchies  are  derived  from  the  task 
analysis  product  and  share  its  important  function  of  defining  the 
foundation  of  a  training  system,  they  rank  in  importance  almost 
equally.  The  desirability  of  an  adequate  and  accurate  task 
analysis  is  emphasized  in  light  of  this  dependent  relationship. 
The  diagram  in  Figure  2  (which  also  appears  in  project  report  no. 
7,  "Task  Analysis  Methodology  Report")  shows  the  relationship 
between  objectives  hierarchy  analysis  products  and  other  instruc¬ 
tional  elements  of  a  training  system. 
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TASK  LISTING  |  |  MAKES  POSSIBLE  >|  CROS  |  |  MAKES  POSSIBLE 
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Figure  2  —  Relations  of  Training  Products  to  Objectives  Hierarchies 
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From  the  diagram  it  should  be  apparent  that  a  proportional 
share  of  the  effects  of  analysis  on  instruction  are  caused  by 
objectives  hierarchy  analysis.  Mainly,  its  influence  is  felt  in 
the  syllabus,  because  all  of  the  syllabus  academic  instructional 
events  derive  from  the  hierarchies. 


6 


3.0  OBJECTIVES  HIERARCHY  ANALYSIS  PROCEDURES 


This  section  presents  a  step-by-step  description  of  the 
procedure  for  hierarchy  analysis  used  for  F-16. 


3.1  Overview 

The  method  described  produces  hierarchies  of  objectives 
beginning  from  stated  performance  tasks  by  extending  the  task 
listing  of  which  the  tasks  are  a  part  to  a  lower  level  of  detail. 
These  lower  level  tasks  are  then  transformed  into  instructional 
objectives  according  to  a  set  of  transformation  rules  applied  in 
an  orderly  sequence. 

As  stated  earlier  in  this  report,  the  beginning  point  of  the 
analysis,  a  completed  task  listing  (sometimes  called  "task  inven¬ 
tory",  "job  task  analysis",  "job  analysis",  and  various  other 
names),  provides  the  individual  tasks  for  which  objectives 
hierarchies  can  be  created.  The  exact  nature  of  the  tasks 
suitable  as  input  to  the  analysis  is  difficult  to  describe  due  to 
the  somewhat  vague  definition  of  "task"  and  the  difficulty 
encountered  by  analysts  in  naming  the  exact  characteristics  of 
tasks  at  any  given  level  or  sub-level. 

The  desirable  point  for  terminating  the  task  analysis 
depends  upon  the  purpose  of  the  analysis.  For  the  purpose  of 
developing  objectives  hierarchies,  task  analysis  may  be  termi¬ 
nated  as  tasks  are  identified  which  constitute  independent, 
unitary  behavior  sequences  of  job  behavior  for  which  precise 
standards  of  measurement  may  be  specified  and  for  which  measure¬ 
ment  of  performance  is  likely  to  be  made  during  training.  (In 
the  F-16  development  procedures,  tasks  which  have  been  assigned 
such  standards  for  measurment  purposes  are  called  criterion- 
referenced  objectives  (CROs).  See  project  report  no.  5,  "Deriva¬ 
tion,  Formating,  and  Use  of  Criterion-referenced  Objective  (CROs) 
and  Criterion-referenced  Tests  (CRTs).")  The  highest  levels  of 
the  task  listing  generally  name  behaviors  so  complex  and  broad 
that  they  cannot  conveniently  be  measured,  nor  can  precise  meas¬ 
urement  standards  be  formulated  for  them.  On  the  other  hand,  at 
the  lowest  levels  of  a  very  detailed  task  listing  the  tasks  found 
are  often  so  minute  that  it  would  not  be  sensible  to  consider 
measuring  them  in  a  real-life  training  situation,  even  though 
very  clear  measurement  standards  are  capable  of  being  written. 
Selecting  tasks  at  this  measurable,  intermediate  level  for  the 
beginning  point  of  analysis  gives  the  benefit  of  defining  the 
major  performance  measurement  "gates"  which  will  become  an 
important  part  of  the  training  course  syllabus.  Deriving 
objectives  starting  at  these  points  relates  the  learning  which 
must  take  place  (stated  in  the  objectives)  directly  to  the 
desired  terminal  performance.  This  becomes  a  tremendous  asset  as 
the  developer  defines  the  instructional  sequence  in  the  syllabus. 


3.2  Procedures 


Transformation  of  the  task  listing  into  an  objectives  hier¬ 
archy  tak  s  place  through  (1)  extension  of  the  task  listing  one 
additionl  level,  (2)  scanning  of  these  extended  tasks  for  parti¬ 
cular  types  of  tasks,  and  (3)  converting  particular  types  of 
tasks  into  instructional  objectives  and  adding  other  objectives 
to  supplement  the  identified  tasks  or  in  some  cases  new  but 
related  tasks. 


3.2.1  Step  1:  A  Task  Is  Selected 

A  task  is  selected  for  analysis  to  begin  with.  If  appro¬ 
priate  measurement  standards  have  not  been  written  previously, 
they  are  written  at  this  time.  In  their  absence,  particular 
constraints  upon  behavior  such  as  timing  or  the  need  for  over¬ 
learning  which  may  affect  an  instructional  objective,  its 
condition,  or  standard,  may  not  be  apparent.  It  is  suggested 
that  larger  more  complex  tasks  be  selected  early  in  analysis  to 
avoid  later  duplications  of  effort.  Eventually  all  tasks,  even 
tasks  at  the  highest  levels  of  the  task  listing  must  be  analyzed. 


3.2.2  Step  2:  Subtasks  Are  Listed 

The  task  listing  for  the  selected  task  is  continued  to  one 
further  level.  This  further  list  of  tasks  provides  the  substance 
for  the  analysis.  The  procedure  for  task  listing  is  described  in 
project  report  no.  7,  "Task  Analysis  Methodology  Report”. 


3.2.3  Step  3:  Decision-making  (Classification  and 

Rule-using)  Behaviors  Are  Identified 

Within  the  newly  extended  list  of  tasks  there  are  possibly 
decision-making  behaviors  which  are  complex  enough  to  require 
isolated  instruction.  The  list  is  scanned  for  these,  and  objec¬ 
tives  are  written  for  them.  The  decision-making  behavior  may 
concern  when  or  how  to  use  a  rule,  which  of  several  rules  or 
procedures  to  use,  or  difficult  discriminations  or  classifica¬ 
tions  which  the  student  must  learn  to  make .  Very  cf ten  instruc¬ 
tion  focuses  inadequately  on  this  variety  of  behavior,  leaving 
the  student  capable  of  carrying  out  procedures  but  incapable  of 
deciding  when  to  use  them  or  incapable  of  making  certain 
decisions  (discriminations  being  an  important  impact)  which 
constitute  a  major  step  in  procedural  execution. 

Most  often  decisions  made  during  performance  of  a  task 
require  only  simple  instructions  to  the  performer  and  do  not 
require  direct,  isolated  instruction  and  practice  in  making  the 
decision.  For  these  decisions  no  objective  needs  to  be  written. 
For  complex  decisions  or  classifications,  however,  an  objective 
should  be  written,  which  will  in  turn  give  rise  to  the  appro¬ 
priate  instructional  presentation. 


The  extended  task  list  itself  should  be  critically  examined 
at  this  time  to  ensure  that  less  readily-visible  decision-making 
behaviors  have  not  been  omitted  through  oversight.  Very  often 
complex  decision-making  behaviors  accomplished  as  second  nature 
by  experienced  persons  are  subtle  and  hard  to  detect  during 
analysis . 


3.2.4  Step  4:  Motor  Control  Behaviors  Are  Identified 

In  the  same  way  that  the  extended  task  list  was  scanned  for 
complex  decision-making  behaviors,  it  is  now  scanned  for  complex 
motor  control  behaviors.  Objectives  of  two  types  are  written  for 
these  behaviors:  (1)  An  objective  calling  for  the  performance  of 
the  behavior  itself,  in  isolation  from  the  other  behaviors 
involved  in  task  execution  and  (2)  an  objective  of  the  type 
described  in  step  5  below  intended  to  establish  the  information 
and  memory  base  requisite  for  performance.  The  informational 
objective  is  prerequisite  to  the  performance  objective,  and 
though  some  will  argue  that  the  two  objectives  are  in  fact  only 
one  objective,  consideration  of  the  instructional  strategy, 
training  device,  and  practice  requirements  implications  of  each 
objective  will  show  that  they  are  in  fact  independent  instruc¬ 
tional  requirements,  each  of  which  should  be  accounted  for  by  its 
own  objective. 


3.2.5  Step  5:  Verbal  Suppport  (Summarization) 

Requirements  Are  Identified 

Summarization  objectives  are  those  which  insure  that  the 
student  is  acquainted  at  the  verbal  level  with  the  sequence, 
parameters,  and  critical  values  associated  with  safe  and  adequate 
task  performance.  The  behavior  in  a  summarization  objective  may 
take  any  one  of  several  forms  requiring  the  student  to  recall, 
identify,  order,  or  otherwise  manipulate  verbal  information 
concerning  task  performance.  For  critical  or  dangerous  tasks, 
exact  recall  should  be  required.  For  other  tasks,  less  rigorous 
forms  of  behavior  may  suffice.  Summarization  objectives  serve 
two  purposes:  (1)  They  provide  the  student  the  information  and 
memory  component  which  are  prerequisite  to  task  performance 
(i.e.,  the  student  cannot  perform  the  task  unless  he  is  told  what 
the  task  is  and  how  to  perform  it)  and  (2)  they  ensure  that  tasks 
on  the  extended  task  listing  become  the  subjects  of  instruction 
even  though  the  tasks  themselves  are  not  complex  or  important 
enough  to  require  isolated  instruction  by  themselves.  At  a  later 
point  in  the  procedure  these  other  tasks  will  be  dropped  as 
separate  entities  from  the  extended  task  list  ("pruned"),  but  the 
summarization  objective  will  keep  them  from  being  lost  to 
instruction.  As  a  rule  a  summarization  objective  should  be  added 
for  each  task  being  analyzed. 


3.2.6  Step  6:  Minor  Tasks  Are  Grouped  (Collocated)  for 

Instruction 

Collocation  is  the  collection  of  tasks  to  one  point  which 
(1)  are  similar  in  intent  and  form  and  differ  only  in  specific 
content  and  (2)  are  not  substantial  enough  to  stand  alone  in 
instruction  but  can  be  combined  with  other  behaviors  closely 
related  in  content  to  form  a  significant  body  of  instruction. 

Many  tasks  in  an  extended  task  listing  are  at  the  level  of  "mani¬ 
pulate  control  X"  or  "locate  control  X".  These  tasks  are  not  of 
sufficient  scope  to  deserve  separate  instruction,  but  the  loca¬ 
tion  of  a  group  of  controls  related  to  one  functional  system  may 
be.  Collocation  gathers  together  tasks  which  are  content-related 
in  this  fashion  and  groups  them  under  an  appropriately-worded 
objective . 


3.2.7  Step  7:  Dangerous  and  Excessively  Expensive 

Behaviors  Are  Scaled  Back 

This  step  is  necessitated  by  the  fact  that: 

a.  Some  behaviors  are  too  dangerous  to  execute  with 
students  during  training. 

b.  Some  behaviors  are  too  expensive  in  time,  resources,  or 
personnel  to  execute  or  simulate  for  training  purposes. 

c.  Some  behaviors  are  too  difficult  to  be  executed  by 
students  at  the  experience  level  they  will  attain  during 
the  projected  course  of  instruction. 

During  this  step  each  task  under  analysis  and  each  entry  in  the 
extended  task  listing  are  examined  to  see  if  any  of  these  condi¬ 
tions  apply.  For  those  tasks  which  are  too  dangerous,  expensive 
or  complicated,  an  objective  must  be  written.  The  objective 
should  state,  however,  the  acceptable  behavioral  level  (perfor¬ 
mance  or  cognitive)  students  must  attain  to  show  that  they  have 
had  sufficient  experience  with  less  dangerous,  less  expensive 
behaviors  to  be  expected  to  perform  adequately  when  the  real 
behavior  is  required. 

3.2.8  Step  8:  Determine  Knowledge-base  Requirements 

Knowledge-base  determining  is  a  process  which  is  carried  out 
for  areas  of  the  hierarchy  in  which  specific  procedures  to  be 
performed  are  not  clearly  defined  or  where  skilled  performers  are 
likely  to  be  required  to  perform  procedures  under  unusual  or 
nonstandard  conditions.  In  situations  where  a  set  procedure  is 
not  prescribed,  or  in  nonstandard  conditions,  students  and 
skilled  job  performers  are  thrown  onto  their  own  judgement  as  to 
how  to  proceed  in  accomplishing  tasks.  In  many  cases  unusual 
conditions  will  force  a  performer  to  modify  procedures  in  a  way 


to  make  them  suitable  for  use  in  the  nonstandard  situation. 
Instructional  developers  cannot  anticipate  these  situations  and 
instruct  performance  under  all  conditions.  Instead,  in  such 
cases  a  substrata  of  information,  concepts,  and  principles  must 
be  provided  which  the  student  can  consult  when  determining  how  to 
modify  standard  procedures  to  meet  unique  conditions. 

Familiarity  with  this  knowledge  base  by  students  is  ensured  by 
incorporating  objectives  appropriately  in  the  hierarchy. 

Knowledge-base  determining  can  increase  the  likelihood  of 
competent  performance.  It  should  also  be  realized  that  the 
addition  of  knowledge  bases  is  an  easily  abused  procedure.  Its 
strength  is  in  producing  job  holders  who  can  act  appropriately  in 
nonstandard  conditions.  Its  weakness  is  that  it  can  easily  add 
to  a  curriculum  large  volumes  of  instruction  of  questionable 
value.  A  tendency  of  the  well-informed  is  to  place  too  much 
value  on  their  information  and  to  appreciate  it  for  its  own  sake. 
Since  the  aim  of  creating  objectives  hierarchies  is  to  define  the 
necessary-and-only  instruction,  the  knowledges  which  are  taught 
only  for  appreciation  value  must  be  eliminated. 

Knowledge-base  determining  is,  unfortunately,  a  somewhat 
intuitive  process  and  must  be  carried  out  with  much  skepticism. 
For  instance,  it  is  commonly  assumed  by  trainers  that  a  knowledge 
base  consisting  of  extensive  machine  mechanical  descriptions  is 
necessary  for  machine  operators.  The  result  of  such  reasoning  is 
usually  a  lengthy  course  on  machine  function  and  engineering 
which  teaches  information  for  which  the  operator  finds  little  or 
no  subsequent  use.  Logically,  it  is  reasonable  that  some  know¬ 
ledge  of  machine  mechanics  would  help  machine  operators  perform 
more  adequately  as  operators,  for  instance  in  responding  to  emer¬ 
gencies.  Knowledge-base  determining  should  seek  to  establish 
those  elements  of  information  which  are  relevant  to  performance 
and  those  which  are  not  and  incorporate  only  those  which  enhance 
the  student's  performance  capability  into  the  hierarchy  of  objec¬ 
tives  as  appropriately  stated  verbal  or  higher-order  knowledge¬ 
using  objectives.  Non-sub jective  principles  for  determining 
knowledge-bases  have  not  been  specified  at  this  point,  but  a 
systematic  procedure  for  doing  it  is  possible  and  will  require 
more  extensive  efforts  than  were  possible  during  the  F-16 
project . 


3.2.9  Step  9:  Remaining  Tasks  Are  Eliminated  from 

Consideration  (Pruning) 

During  pruning,  tasks  are  dropped  from  consideration  which 
do  not  require  individual  coverage  in  instruction  as  the  subject 
of  one  instructional  sequence.  Along  with  identifying  major, 
trainable  behaviors,  task  listing  generally  identifies  details  of 
task  performance  which  are  not  large  enough  bodies  of  behavior  to 
require  separate  instruction.  These  are  prevented  from  being 
lost  by  the  previous  activities  of  summarizing  (step  5)  and 
collocation  (step  6).  Pruning  consists  of  a  final  examination  of 


the  extended  task  listing  to  determine  whether  major  behaviors 
(cognitive,  psychomotor,  or  affective)  which  are  of  trainable 
scope  have  been  overlooked  and  require  objectives  to  be  written. 
Any  such  objectives  are  added  at  this  time. 

Additionally,  it  may  be  observed  at  this  time  that  other 
tasks  listed  in  Step  2  are  themselves  complex  or  extensive  enough 
to  merit  the  task  listing  process.  That  need  should  be  noted 
during  pruning.  The  task  in  question  should  be  turned  into  a 
performance  objective  (a  new  task  in  the  task  listing),  and  the 
analysis  procedure  should  be  applied  to  it  in  its  turn. 


3.2.10  Step  10:  Cycle  to  a  New  Task 

With  analysis  of  the  original  task  being  complete,  the 
process  may  cycle  on  to  a  new  task  until  all  tasks  have  undergone 
analysis.  This  must  include,  as  was  mentioned  earlier,  examina¬ 
tion  of  higher-level  tasks  in  the  task  listing  which  very  often 
require  decision-making  or  summarization  objectives  to  be 
written . 


3 . 3  Limitations 

Some  argument  may  be  made  with  the  analysis  method  described 
above.  An  attempt  is  made  here  to  answer  some  anticipated 
objections . 

3.3.1  Built-in  Assumptions 

The  reader  familiar  with  analysis  will  recognize  several 
assumptions  built  into  the  method  described.  For  example,  the 
assumptions  are  made  that  instruction  at  the  verbal  level  should 
precede  instruction  and  practice  at  the  performance  level  and 
that  instructional  objectives  should  be  of  sufficient  substance 
and  importance  to  merit  independent,  specifically  designed 
instructional  presentations.  To  the  extent  that  the  reader 
disagrees  with  those  or  any  other  assumptions,  that  disagreement 
will  constitute  a  limitation  of  this  method. 


3.3.2  Proceduralization 

It  will  occur  to  some  that  being  so  specific  and  deliberate 
about  the  act  of  analysis  and  suggesting  that  nonscientists  may 
participate  in  analyses  will  produce  a  host  of  inadequate  and 
shallow  analyses.  That  is  inevitably  true,  but  it  is  true  of  any 
method  of  analysis.  A  host  of  inadequate  and  shallow  analyses 
will  result  from  the  indiscriminate  and  thoughtless  application 
of  any  analysis  procedure  (or  nonprocedure)  in  any  hands. 


w 
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Analysis  is  a  process  of  guided  thinking,  decisions-making , 
and  representation  of  decisions  in  recorded  form.  A  "method"  of 
analysis  can  be  no  more  than  a  guide  to  thinking,  and  never  a 
substitute  for  thought.  If  only  the  high  degree  of  procedurali- 
zation  attending  this  method  is  relied  upon  to  produce  quality, 
bad  analyses  will  result.  On  the  other  hand,  if  the  procedural 
aspects  of  the  analysis  remind  the  analyst  to  consider  worthwhile 
objectives  which  otherwise  might  have  been  omitted,  then  the 
procedure  functions  as  a  strengthening  supplement  to  the  analysis 
and  is  very  desirable.  It  is  the  intention  that  this  be  the 
case . 
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4.0  OBJECTIVES  HIERARCHY  ANALYSIS  UPDATE 


This  section  repeats  briefly  a  position  already  stated  in 
project  report  no.  7,  "Task  Analysis  Methodology  Report."  That 
position  places  emphasis  on  the  importance  of  maintaining  all  of 
the  foundation  documents  of  the  F-16  instructional  system  in  a 
continuously  updated  condition  so  that  the  system  may  be  kept 
current  and  efficient. 


4.1  Procedure  for  Objectives  Hierarchy  Update 

A  procedure  for  regular  objectives  hierarchy  update  is 
contained  in  project  report  no.  10,  "Data  Base  Update  Proce¬ 
dures  .  " 


4.2  Organizational  Responsibilities  for  Objectives  Hierarchy 

Maintenance 

Organizational  responsibilities  for  maintenance  of  the 
objectives  hierarchies  are  nearly  identical  to  those  for  task 
listing  maintenance.  (See  Table  1.)  Personnel  responsible  for 
task  listing  maintenance  are  also  responsible  for  objectives 
hierarchies  maintenance.  Remarks  pertaining  to  the  need  for 
training  of  responsible  personnel  apply  to  objectives  hierarchies 
as  well  as  task  listings. 

4.3  Support  Requirements  for  Objectives  Hierarchy  Maintenance 

Those  requirements  listed  in  project  report  no.  7  as 
requsites  for  task  listing  maintenance  are  also  requisites  for 
objectives  hierarchy  maintenance. 


TABLE  1 — Organizational  roles  for  F-16 
objectives  hierarchy  (OH). 


Organization/ Per son 

OTDT/OH  Specialist  (same 
person  as  TL  specialist) 


OTDT/Leader 


Responsibility 

1.  Identify  changes  to  TL  or 
data  from  evaulation  influ¬ 
encing  need  for  update. 

2.  Execute  OH  changes  as 
required  by  incoming 
information . 

3 .  Conduct  content  and  tech¬ 
nique  validation  of  the  OH 
with  extra-OTDT  instruc¬ 
tional  development  special¬ 
ists  . 

4.  Obtain  agreement  of  OTDT 
and  other  controlling 
agencies  on  OH  changes. 

5.  Oversee  insertion  of 
changes  into  OH  data  base. 

6.  Produce  new  OH  versions  and 
disseminate  to  appropriate 
persons . 

1 .  Coordinate  and  arrange 
extra-OTDT  OH  assistance 
contacts,  such  as  experts 
for  validation  and  informa¬ 
tion-producing  organiza¬ 
tions  . 


Ensure  use  of  standard  OH 
modification  procedures. 

Train  new  OTDT  OH  special¬ 
ists  or  obtain  training  for 
them. 


TAC  HQ/DOOS 


4 .  Approve  OH  changes  at  OTDT 
level . 

1.  Supply  OTDT  with  current 
information  regarding  con¬ 
cept  of  aircraft  employment 
and  desired  training  empha- 
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T^ble  1 — (cont.) 


Organization/ Person 


4444th  OS  (Luke) 


Responsibility 

2.  Supply  command  emphasis 

necessary  to  establish  and 
keep  open  direct  informa¬ 
tion  channe 1 s . 

1.  Periodically  conduct  review 
of  F-16  TL  to  insure 
updated  OH  condition  and 
observance  of  standard  OH 
modification  procedures. 

2.  Supervise  dissemination  of 
updated  OH  documents . 
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